Repository navigation
Decode the 5/MG optical and raw-IMU historical layouts on Android - #2018
Merged
Merged
Conversation
Base automatically changed from
fix/1992-frontier-and-layout-warning
to
main
September 9, 2026 07:38
…outs v20 (raw optical) and v21 (raw 6-axis IMU) decoded to null on Android while Swift dispatched both. The decoders were already here: `Whoop5RawOptical` and `Whoop5RawImu` were wired to the LIVE deep-buffer route and never to the type-47 historical dispatch, so an offloaded record in either layout yielded nothing at all, not even the timestamp its shared v18 header carries. `decodeWhoop5HistoricalV2021` is the port. Both layouts reuse that header, so the layout marker, the monotonic record index and the unix now decode for records that previously produced none. v21 emits its six 100-sample i16 channels raw and unscaled, exactly as Swift does, with the physical scales left to `Whoop5RawImu.decode`; a channel is emitted only at a full 100 samples, so a truncated record cannot produce an array that looks complete. v20 goes through `Whoop5RawOptical` for the block headers, per-block sample counts and channel arrays, asserting no wavelength and no absolute unit. This does NOT make those nights stageable, and is not meant to. Neither layout carries a per-second heart rate or a gravity vector; they carry raw channels no engine reads, and mapping those to a physiological value is the open work on #1992. What it buys is the same decode on both platforms, a real timestamp and record index where there was nothing, and the arrays available on the device. The records are still archived, which is the part that had to be got right in the correct order. The reject filter's decode-outcome test was aligned with Swift's first, so it asks whether a record yielded a unix AND either a heart rate or gravity rather than whether it decoded at all. Without that already in place this commit would have silently stopped preserving the very bytes the channel mapping needs, on the only platform collecting them. `whoop5V20StillArchivedAfterItDecodes` and its v21 twin pin exactly that, and they fail against the old filter. Two tests flip rather than break: the mapped-set lockstep now reads {18, 20, 21, 26}, and the layout classifier now answers DECODES_WITHOUT_NAMED_SIGNAL for v20 where it answered UNMAPPED. Both were written to flip here. Independently corroborated while checking this: OpenStrap's Dart decoder (MIT) reads v20's geometry the same way we do, body start, block stride, slot offsets and sign-extended 20-bit samples. No code taken; the agreement is evidence, not a source. Android suites 5,752 green.
…ader The tests shipped with the port used a short v18-derived frame with its version byte changed. On one of those `Whoop5RawOptical.decode` refuses outright, because it requires the frame to BE a 2140-byte buffer, so only the shared header decoded and the optical branch the port exists for was never run at all. Every assertion passed and none of them touched it. There is now an end-to-end case on a real-shaped, checksum-sealed 2140-byte v20 buffer: the block count, the block's sample count, the widest-block sample total, the channel-present tally, the raw header, and a planted i32 sample read back unmasked and unscaled through the historical dispatch. It also asserts the record is still archived, so the guard holds on a frame that decodes fully rather than only on one that stops at the header. Also checked while re-reviewing, and unchanged: `decodeHistorical` now returning non-null for v20 and v21 does NOT move the completed-offload classification. `decodedChunksThisSession` counts non-empty StreamBatch commits, and these layouts still produce no rows, so a strap banking only them is still classified as having banked no sensor history and still says so. Android suites 5,753 green.
ryanbr
force-pushed
the
feat/1992-android-v20-v21
branch
from
September 9, 2026 07:42
6d35fc1 to
88ee566
Compare
DX23876
added a commit
to DX23876/noop
that referenced
this pull request
Sep 16, 2026
Thirty real conflicts. The Android tree is dropped again per FORK_GUIDE; every other hunk was resolved by asking which mechanism this fork actually runs, not by preferring a side. Taken from upstream: • Protocol: ryanbr#2017/ryanbr#2018/ryanbr#2020 — the 5/MG historical optical and raw-IMU layouts, and the v26 window's ABSOLUTE base code beside its deltas. Its migration lands as `v57-ppg-waveform-base-code` (upstream's v44), since this fork's sequence is nine ahead of theirs. • BLE diagnostics ryanbr#1998-ryanbr#2003: a bond end that says WHY, named BLE integers, a session-budgeted hex dump, and "we hold the link" only when the OS says so. • Battery: ryanbr#1951 stop asking a 5/MG for a reading it refuses; ryanbr#1959 poll 0x2A19 for an unbonded one. • Scoring: ryanbr#1978/ryanbr#1982/ryanbr#1990 the RHR/HRV baseline gates, ryanbr#1977 the strain denominator's domain, RMSSD withheld on an over-counted night, ryanbr#1943 the resting-HR bin gate, the closed final HR/HRV window, HR-recovery requiring one CONTINUOUS effort, ryanbr#1984 nights bridged under minSleepMin, ryanbr#2015 motion re-folded only when that day's gravity moved, ryanbr#2016 a provisional Rest. • Charts ryanbr#2004/ryanbr#2005/ryanbr#2007/ryanbr#2009/ryanbr#2011 and ryanbr#1995's hero-ring detail route, ryanbr#1164's "Pending sync" caption under Rest. • Widgets: the heart-rate widget with its trace (ryanbr#1957/ryanbr#1964) and the widget cost counters. • Health: ryanbr#1503 — the Apple Health external UUID drops the strap id, so a re-pair no longer strands every prior record, plus the one-off sweep that reclaims the stranded ones; ryanbr#1205 GPS routes read back from Apple Health. • Storage: rolling retention for ppgWaveformSample, the skin-temp absolute backfill outside the 21-night window, per-table byte estimates. • Units split (body vs exercise distance), per-day wake overrides, the wind-down nudge pinned to its own day, and the German Sie→du conversion (29 entries, merged per LOCALE so the fork's own de edits stayed). Kept on the fork's side, with the reason: • ryanbr#1005's in-memory `dayScanCache` stays out, again: the persisted `DayScanFingerprint` survives relaunch and its key folds the per-day weight and max HR upstream's does not. • The steps-motion cache stays PERSISTED for the same reason. Upstream's failed-read-vs-empty-day distinction IS adopted into it — on a persisted cache a stale zero from a read error would outlive the process that made it. • Every stream read in the day loop keeps `limit: Int.max`. All five are uncapped together; taking StreamReadCap for three of them would leave the loop half-capped, which is worse than either choice. • The coach engine. Upstream's K1-K15 layer is not adopted: its single 40-row `coachMessage` table, its tool-less pre-baked context and its own brief are behind what this fork already ships. Its ADDITIVE half is kept — the on-device voice input, the Siri "Ask Coach" intent, the brief scheduler and brief widget, and streaming for the OpenAI-compatible providers this fork's Anthropic-only StreamingToolClient does not cover. They are wired up in the next commit; `CoachBriefWidget` is deliberately not registered until then. • Liquid's `DomainTheme` labels, value-sampled Charge tint, `LiquidScoreGauge` and `TodayCardSurface`; classic Today's ring order and `LiveEffort.today`. • RootTabView's four-tab shell and `MoreCatalog`: Coach hangs on content here, not on a tab. Two duplicate mechanisms collapsed rather than carried: • `ChargeBreakdownWiring` (upstream) and `ChargeBreakdownFormat` (fork) are the same extraction done twice. The fork's survives, Coupled now reads it, and upstream's TEST is kept — it is coverage this path never had. • Upstream's `CoachMessageStore` is dropped; `CoachConversationStore` already autosaves the whole multi-conversation history. One defect found while resolving, fixed here: `LiveEffort.today` read HR with no explicit limit, so `hrSamples`' 8000-row default truncated it — at ~18k rows a day that covered about ten hours and the live Effort silently stopped climbing. It failed safe (`effectiveEffort` takes the max), which is why it went unseen. `Live5RestFrozenTests` is removed by upstream, which replaces it with `Live5NoGravityRestChainTests`. `local_day_windows_oracle.json` moves out of upstream's android path into `Tests/StrandAnalyticsTests/Oracles/` — NOT `Fixtures/`, which this repo's .gitignore excludes everywhere. Verified: StrandAnalytics 2233, WhoopStore 627, WhoopProtocol 700, StrandImport 293, StrandDesign 120, NoopLocalAccess 14, SemanticMemory 19, OuraProtocol 209 and PolarProtocol 25 all green; macOS and iOS both build; doc_comment_lint clean; `i18n_audit --ci` byte-identical to its pre-merge output (it is red on main for reasons that predate this, and the seven new widget literals were given all nine locales so the gate did not get worse). Six StrandTests were ALREADY failing on main before this merge and still are — `CoachMyLogsAndZonesTests` (the tool census pin was not bumped when the Hevy tools landed) and five in `RawPhysiologyUnionTests`. Every input to them — CoachTools.swift, ToolConsent.swift, Repository.swift and both test files — is byte-identical to main, so this merge neither caused nor fixed them. They went unnoticed because app-build.yml last ran on 2026-08-25. Analysis migration required: YES. The scoring changes above make stored derived values stale; the recipe bump and its resumable migration are the next commit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Was stacked on #2017, which has since merged; this now targets main and the branch was rebuilt on it,
so the diff is just the port. It still DEPENDS on what #2017 landed: the archive's decode-outcome test
now asks whether a record yielded something scoreable rather than whether it decoded, which is the
only thing that keeps a v20 record in the reject archive once it starts decoding.
Note this had NO CI while it was stacked. Every workflow is
pull_request: branches: [main], so a PRbased on a branch gets zero checks rather than green ones. It is only now being verified.
What
v20 (raw optical) and v21 (raw 6-axis IMU) decoded to null on Android while Swift dispatched both. The
decoders were already here:
Whoop5RawOpticalandWhoop5RawImuwere wired to the LIVE deep-bufferroute and never to the type-47 historical dispatch, so an offloaded record in either layout yielded
nothing at all, not even the timestamp its shared v18 header carries.
decodeWhoop5HistoricalV2021is the port of Swift's function of the same shape. Both layouts reusethe v18 record header, so the layout marker, monotonic record index and unix now decode. v21 emits its
six 100-sample i16 channels raw and unscaled exactly as Swift does, with the physical scales left to
Whoop5RawImu.decode, and only at a full 100 samples so a truncated record cannot produce an arraythat looks complete. v20 goes through
Whoop5RawOpticalfor block headers, per-block sample countsand channel arrays, asserting no wavelength and no absolute unit.
What this does NOT do
It does not make those nights stageable, and is not meant to. Neither layout carries a per-second
heart rate or a gravity vector; they carry raw channels no engine reads. Mapping those channels to a
physiological value is the open work on #1992 and still needs a capture. What this buys is the same
decode on both platforms, a real timestamp and record index where there was nothing, and the arrays
available on the device.
The ordering mattered
The records must still reach the reject archive. v20 is no longer archived by the unmapped-layout rule
(it is mapped now) nor for failing to decode (it decodes now), so the only thing keeping it is the
decode-outcome test asking whether a record yielded a unix AND either a heart rate or gravity. #2017
aligned that test with Swift's first, deliberately as a no-op at the time.
Without that already in place, this commit would have silently stopped preserving the very bytes the
channel mapping needs, on the only platform collecting them.
whoop5V20StillArchivedAfterItDecodesand its v21 twin pin it, and they fail against the old filter.
Tests that flip rather than break
The mapped-set lockstep now reads
{18, 20, 21, 26}, and the layout classifier answersDECODES_WITHOUT_NAMED_SIGNALfor v20 where it answeredUNMAPPED. Both were written in #2017 toflip exactly here, so the change shows up as two repins rather than as a surprise.
Provenance
Ported from this repository's own Swift decoder. While checking it I compared against OpenStrap's Dart
decoder (MIT), which reads v20's geometry the same way: body start, block stride, slot offsets and
sign-extended 20-bit samples. No code taken from it; the agreement is corroborating evidence for our
reading, not a source for it.
Verification
Tools/doc_comment_lint.pyclean.carry no per-second heart rate and no motion" instead of "doesn't decode yet", and the reject archive
should keep collecting the same records it did before.